Skip to content

feat(scanner): add enterprise data protection and database rules - #278

Open
TFT444 wants to merge 7 commits into
devfrom
feat/261-storage-protection
Open

feat(scanner): add enterprise data protection and database rules#278
TFT444 wants to merge 7 commits into
devfrom
feat/261-storage-protection

Conversation

@TFT444

@TFT444 TFT444 commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements issue #261 as a policy-driven Azure data-protection and database-security rule pack.

Added controls

  • Storage shared-key authorization and minimum TLS enforcement.
  • Opt-in customer-managed-key and blob immutability checks.
  • SQL Microsoft Entra-only authentication.
  • SQL vulnerability-assessment configuration.
  • SQL audit-retention minimum (90 days).
  • Opt-in Cosmos DB local-authentication and public-network checks.
  • Opt-in managed-cache public-access/TLS checks.

Implementation

  • Added Azure client accessors for SQL vulnerability assessment, Cosmos DB accounts, and managed caches.
  • Added conservative opt-in policy tags and explicit exception handling.
  • Added rule documentation, framework mappings, rule-reference updates, and CLI playbooks.
  • Added azure-mgmt-cosmosdb==10.0.0 and azure-mgmt-redis==14.5.0.

Validation

  • Focused Implement enterprise data protection and database security rules #261/storage tests: 32 passed.
  • Ruff: passed.
  • Python compilation: passed.
  • Framework JSON validation: passed.
  • git diff --check: passed.
  • Full local suite: 585 passed, 2 skipped; 4 unrelated failures remain:
    • 2 pre-existing Chroma dependency/environment failures.
    • 2 unrelated untracked az_cmp/az_net tests.

The playbooks are review-safe guidance commands; they do not apply irreversible Azure changes automatically.

Closes #261

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@TFT444
TFT444 requested review from m-khan-97 and removed request for SHAURYAKSHARMA24 and parthrohit22 August 17, 2026 01:20
@TFT444 TFT444 self-assigned this Aug 17, 2026
@Vishnu2707

Copy link
Copy Markdown
Member

@ritiksah141 @parthrohit22 - an initial review would be appreciated.

@m-khan-97 m-khan-97 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this rule pack together. I went through the implementation and tests carefully. The direction is useful, but I found three things we need to fix before merging:

  1. AZ-STOR-009 cannot currently be enabled for a real blob container. The shared policy helper reads resource.tags, while Azure BlobContainer objects expose metadata rather than ARM tags. That means the documented oshield:immutability-required marker will never be seen and the rule will silently skip every real container. Please use a supported source of applicability, such as container metadata or an account-level policy, and add a regression test with an SDK-shaped BlobContainer object.

  2. The NIST mapping for AZ-DB-007 is incorrect. A.12.4.1 is an ISO 27001 control, not a NIST CSF identifier. Please correct it consistently in the rule, nist_csf.json, and the rules reference. If there is no verified direct mapping, use an explicit N/A value.

  3. Several files presented as remediation playbooks only print guidance. This affects the cache, Cosmos, and SQL rules in this PR. Since the API exposes these files as CLI remediation, either implement safe, validated commands with proper argument handling or classify them clearly as manual guidance instead of executable remediation.

I ran the focused rule tests locally and all 24 passed. The Azure client test module could not collect because this checkout is missing azure.mgmt.authorization, so I did not treat that environment issue as a PR failure. The points above are production contract gaps that the current mocked tests do not cover.

Once these are addressed, I will be happy to take another look.

@TFT444 TFT444 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three blockers addressed in the latest commit:

  1. AZ-STOR-009 opt-in check moved from the BlobContainer to the parent storage account, which exposes ARM tags via the SDK. All containers under a tagged account are now evaluated.
  2. AZ-DB-007 NIST mapping corrected from A.12.4.1 (ISO 27001) to PR.PT-1 across the rule, nist_csf.json, and rules-reference.
  3. All six echo-only playbooks now contain validated az CLI commands behind an APPLY confirmation gate.

Ready for re-review.

@TFT444

TFT444 commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

@m-khan-97 all three points from your review are addressed. AZ-STOR-009 opt-in now checks both the parent account and the individual container (OR logic, so account-level tagging still covers all containers). AZ-DB-007 NIST mapping corrected to PR.PT-1 across the rule, nist_csf.json, and rules reference. All six echo-only playbooks now contain validated az CLI commands with a confirmation gate. Ready for re-review when you have time.

@m-khan-97

Copy link
Copy Markdown
Collaborator

@TFT444, thanks for addressing the earlier findings. This branch now conflicts with dev and is substantially behind it. Please rebase cleanly onto current dev and remove the temporary ChromaDB CVE audit exclusions, since #317 has resolved that dependency issue at source. Rerun the full suite afterward, then request my review on the new head.

@TFT444

TFT444 commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

@TFT444, thanks for addressing the earlier findings. This branch now conflicts with dev and is substantially behind it. Please rebase cleanly onto current dev and remove the temporary ChromaDB CVE audit exclusions, since #317 has resolved that dependency issue at source. Rerun the full suite afterward, then request my review on the new head.

@m-khan-97 Conflict have been solved please have a look

@openshield-org openshield-org deleted a comment from ritiksah141 Aug 28, 2026
@m-khan-97

Copy link
Copy Markdown
Collaborator

@TFT444, I checked the new head directly. The merge conflict is resolved and all 20 checks pass, but one requested cleanup is still missing: .github/workflows/ci.yml on this branch still adds --ignore-vuln CVE-2026-45830 and CVE-2026-45833. Current dev no longer contains those exclusions, and #317 removed ChromaDB from the core requirements, so carrying them back would silently weaken the dependency gate for no remaining reason.

Please remove both exclusions and rerun CI. The three substantive rule/playbook fixes are still present after the merge; I will complete their final source review once the security-gate regression is removed.

@TFT444

TFT444 commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

@m-khan-97 all three items from your review are addressed in the current HEAD: AZ-STOR-009 now checks both the parent account and container for the opt-in tag, AZ-DB-007 NIST mapping is corrected to PR.PT-1, and all six playbooks run validated az CLI commands behind an APPLY confirmation gate. CI is fully green. Could you re-review when you get a chance?

@TFT444

TFT444 commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator Author

@parthrohit22 check the pr please require your approval i believe its good to go

ritiksah141
ritiksah141 previously approved these changes Aug 29, 2026

@ritiksah141 ritiksah141 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good

@parthrohit22 parthrohit22 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work on this rule pack. The scope and opt-in policy model are useful, and the earlier review points appear to be addressed. I found four remaining blockers before I can approve.

  1. AZ-STOR-006 skips the insecure Shared Key default.

    The rule treats allow_shared_key_access=None as unknown and emits no finding. Azure documents null for this property as equivalent to allowing Shared Key authorization, so default-configured Storage accounts are currently missed.

    Please treat None as enabled for this rule and add a regression test for that SDK-shaped value.

  2. AZ-STOR-007 skips the insecure TLS default and does not normalize SDK enums.

    Azure documents an unset Storage minimum_tls_version as TLS 1.0. The rule currently skips None, which suppresses a finding for an insecure default. The SDK property can also be a MinimumTlsVersion enum, but the rule uses str(value) rather than the existing enum_str() helper.

    Please normalize the value with enum_str() and treat an unset value as TLS 1.0. Add tests for both None and an enum-shaped TLS 1.0 value.

  3. AZ-STOR-008’s CMK playbook parses a standard Key Vault key URI incorrectly.

    For a URI such as:

    https://<vault>.vault.azure.net/keys/<key-name>/<version>
    

    the script passes https://<vault>.vault.azure.net/keys/<key-name> as the vault URI and <version> as the key name. Azure CLI requires the vault URI, key name, and optional key version separately.

    Please either accept these as separate parameters or parse the URI correctly and pass --encryption-key-version when a version is supplied. Add a shell-level regression test or equivalent command-construction test.

  4. Remove the two reintroduced pip-audit exclusions.

    This PR adds exclusions for CVE-2026-45830 and CVE-2026-45833 in .github/workflows/ci.yml. These are unrelated to the rule pack and weaken the dependency security gate. Please remove them before merge.

Once these items are addressed and the full CI suite is green on the new head, I will re-review promptly.

TFT444 added a commit that referenced this pull request Aug 31, 2026
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
@TFT444

TFT444 commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator Author

@m-khan-97 @ritiksah141 @parthrohit22 All the request have been fixed and sort it out i believe it good to go please double check guys .

Thank You

ritiksah141
ritiksah141 previously approved these changes Sep 1, 2026
parthrohit22
parthrohit22 previously approved these changes Sep 1, 2026

@parthrohit22 parthrohit22 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 4 items from my last review are fixed on the current head:

  • AZ-STOR-006: None now treated as insecure (matches Azure's documented default), only False is compliant.
  • AZ-STOR-007: None treated as TLS 1.0, and the SDK enum goes through enum_str() instead of raw str().
  • AZ-STOR-008 playbook: verified the URI-parsing fix directly in bash — https://vault.../keys/name/version now splits into the three separate --encryption-key-vault/--encryption-key-name/--encryption-key-version args correctly, with and without a version segment.
  • The two unrelated pip-audit CVE exclusions are gone.

Ran tests/test_rules_storage.py myself: 22/22 pass, plus bash -n on the playbook.

Approving.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…ules

- Move AZ-STOR-009 opt-in check from BlobContainer (no ARM tags) to
  the parent storage account, which exposes tags via the SDK; all
  containers under a tagged account are now evaluated for immutability.
- Replace incorrect NIST mapping A.12.4.1 (ISO 27001) on AZ-DB-007
  with PR.PT-1 across az_db_007.py, nist_csf.json, and rules-reference.
- Add executable az CLI commands to fix_az_cache_001, fix_az_cosmos_001,
  fix_az_cosmos_002, fix_az_db_005, fix_az_db_006, and fix_az_db_007
  playbooks; each validates the target and requires APPLY confirmation
  before modifying any Azure resource.
- Update storage-protection-controls.md to document the account-level
  tagging scope for AZ-STOR-009.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…OR-009)

The policy_required guard was placed at the account level, but the
oshield:immutability-required tag is set per container. Moving the
check inside the container loop allows containers with the tag to be
evaluated regardless of whether the parent account carries it.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…AZ-STOR-009)

The policy_required guard was placed at the account level only, but the
oshield:immutability-required tag may be set per-container or per-account.

Now uses OR logic: a container is evaluated if the account carries the
requirement tag (protecting all containers) OR if the container itself
carries it (per-container opt-in). Both cases were previously broken:
the account-level check did not reach container-tagged resources, and
no per-container check existed at all.

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…7/008

- AZ-STOR-006: treat allow_shared_key_access=None as insecure (Azure
  documents unset as equivalent to True); only False is compliant
- AZ-STOR-007: treat minimum_tls_version=None as TLS 1.0 (Azure default);
  use enum_str() instead of str() to handle SDK enum objects correctly
- AZ-STOR-008 playbook: fix Key Vault URI parsing; the previous bash
  expansion passed the wrong segments to --encryption-key-vault and
  --encryption-key-name; now splits vault URI, key name, and optional
  key version correctly
- ci.yml: remove CVE-2026-45830 and CVE-2026-45833 pip-audit exclusions
  (chromadb CVEs unrelated to this PR; resolved by PR #317)

Adds regression tests for None-as-default behavior and SDK enum handling
in AZ-STOR-006 and AZ-STOR-007 (22 storage tests, all passing).

Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
@TFT444
TFT444 dismissed stale reviews from parthrohit22 and ritiksah141 via ca34fb6 September 1, 2026 15:20
@TFT444
TFT444 force-pushed the feat/261-storage-protection branch from a7f180d to ca34fb6 Compare September 1, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement enterprise data protection and database security rules

5 participants